{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.3～3.6 ネットワークモデルの作成\n",
    "\n",
    "- 本ファイルでは、PSPNetのネットワークモデルと順伝搬forward関数を作成します。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.3 学習目標\n",
    "\n",
    "1.\tPSPNetのネットワーク構造をモジュール単位で理解する\n",
    "2.\tPSPNetを構成する各モジュールの役割を理解する\n",
    "3.\tPSPNetのネットワーククラスの実装を理解する\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.4 学習目標\n",
    "\n",
    "1.\tFeatureモジュールのサブネットワーク構成を理解する\n",
    "2.\tサブネットワークFeatureMap_convolution を実装できるようになる\n",
    "3.\tResidual Blockを理解する\n",
    "4.\tDilated Convolutionを理解する\n",
    "5.\tサブネットワークbottleNeckPSPとbottleNeckIdentifyPSPを実装できるようになる\n",
    "6.\tFeatureモジュールを実装できるようになる\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.5 学習目標\n",
    "\n",
    "1.\tPyramid Poolingモジュールのサブネットワーク構成を理解する\n",
    "2.\tPyramid Poolingモジュールのマルチスケール処理の実現方法を理解する\n",
    "3.\tPyramid Poolingモジュールを実装できるようになる\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.6 学習目標\n",
    "\n",
    "1.\tDecoderモジュールのサブネットワーク構成を理解する\n",
    "2.\tDecoder モジュールを実装できるようになる\n",
    "3.\tAuxLossモジュールのサブネットワーク構成を理解する\n",
    "4.\tAuxLossモジュールを実装できるようになる\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 事前準備\n",
    "\n",
    "\n",
    "とくになし"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# パッケージのimport\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.3 PSPNetのネットワーク構造"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PSPNet(nn.Module):\n",
    "    def __init__(self, n_classes):\n",
    "        super(PSPNet, self).__init__()\n",
    "\n",
    "        # パラメータ設定\n",
    "        block_config = [3, 4, 6, 3]  # resnet50\n",
    "        img_size = 475\n",
    "        img_size_8 = 60  # img_sizeの1/8に\n",
    "\n",
    "        # 4つのモジュールを構成するサブネットワークの用意\n",
    "        self.feature_conv = FeatureMap_convolution()\n",
    "        self.feature_res_1 = ResidualBlockPSP(\n",
    "            n_blocks=block_config[0], in_channels=128, mid_channels=64, out_channels=256, stride=1, dilation=1)\n",
    "        self.feature_res_2 = ResidualBlockPSP(\n",
    "            n_blocks=block_config[1], in_channels=256, mid_channels=128, out_channels=512, stride=2, dilation=1)\n",
    "        self.feature_dilated_res_1 = ResidualBlockPSP(\n",
    "            n_blocks=block_config[2], in_channels=512, mid_channels=256, out_channels=1024, stride=1, dilation=2)\n",
    "        self.feature_dilated_res_2 = ResidualBlockPSP(\n",
    "            n_blocks=block_config[3], in_channels=1024, mid_channels=512, out_channels=2048, stride=1, dilation=4)\n",
    "\n",
    "        self.pyramid_pooling = PyramidPooling(in_channels=2048, pool_sizes=[\n",
    "            6, 3, 2, 1], height=img_size_8, width=img_size_8)\n",
    "\n",
    "        self.decode_feature = DecodePSPFeature(\n",
    "            height=img_size, width=img_size, n_classes=n_classes)\n",
    "\n",
    "        self.aux = AuxiliaryPSPlayers(\n",
    "            in_channels=1024, height=img_size, width=img_size, n_classes=n_classes)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.feature_conv(x)\n",
    "        x = self.feature_res_1(x)\n",
    "        x = self.feature_res_2(x)\n",
    "        x = self.feature_dilated_res_1(x)\n",
    "\n",
    "        output_aux = self.aux(x)  # Featureモジュールの途中をAuxモジュールへ\n",
    "\n",
    "        x = self.feature_dilated_res_2(x)\n",
    "\n",
    "        x = self.pyramid_pooling(x)\n",
    "        output = self.decode_feature(x)\n",
    "\n",
    "        return (output, output_aux)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.4 Featureモジュール"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class conv2DBatchNormRelu(nn.Module):\n",
    "    def __init__(self, in_channels, out_channels, kernel_size, stride, padding, dilation, bias):\n",
    "        super(conv2DBatchNormRelu, self).__init__()\n",
    "        self.conv = nn.Conv2d(in_channels, out_channels,\n",
    "                              kernel_size, stride, padding, dilation, bias=bias)\n",
    "        self.batchnorm = nn.BatchNorm2d(out_channels)\n",
    "        self.relu = nn.ReLU(inplace=True)\n",
    "        # inplase設定で入力を保存せずに出力を計算し、メモリ削減する\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.conv(x)\n",
    "        x = self.batchnorm(x)\n",
    "        outputs = self.relu(x)\n",
    "\n",
    "        return outputs\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "class FeatureMap_convolution(nn.Module):\n",
    "    def __init__(self):\n",
    "        '''構成するネットワークを用意'''\n",
    "        super(FeatureMap_convolution, self).__init__()\n",
    "\n",
    "        # 畳み込み層1\n",
    "        in_channels, out_channels, kernel_size, stride, padding, dilation, bias = 3, 64, 3, 2, 1, 1, False\n",
    "        self.cbnr_1 = conv2DBatchNormRelu(\n",
    "            in_channels, out_channels, kernel_size, stride, padding, dilation, bias)\n",
    "\n",
    "        # 畳み込み層2\n",
    "        in_channels, out_channels, kernel_size, stride, padding, dilation, bias = 64, 64, 3, 1, 1, 1, False\n",
    "        self.cbnr_2 = conv2DBatchNormRelu(\n",
    "            in_channels, out_channels, kernel_size, stride, padding, dilation, bias)\n",
    "\n",
    "        # 畳み込み層3\n",
    "        in_channels, out_channels, kernel_size, stride, padding, dilation, bias = 64, 128, 3, 1, 1, 1, False\n",
    "        self.cbnr_3 = conv2DBatchNormRelu(\n",
    "            in_channels, out_channels, kernel_size, stride, padding, dilation, bias)\n",
    "\n",
    "        # MaxPooling層\n",
    "        self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.cbnr_1(x)\n",
    "        x = self.cbnr_2(x)\n",
    "        x = self.cbnr_3(x)\n",
    "        outputs = self.maxpool(x)\n",
    "        return outputs\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ResidualBlockPSP(nn.Sequential):\n",
    "    def __init__(self, n_blocks, in_channels, mid_channels, out_channels, stride, dilation):\n",
    "        super(ResidualBlockPSP, self).__init__()\n",
    "\n",
    "        # bottleNeckPSPの用意\n",
    "        self.add_module(\n",
    "            \"block1\",\n",
    "            bottleNeckPSP(in_channels, mid_channels,\n",
    "                          out_channels, stride, dilation)\n",
    "        )\n",
    "\n",
    "        # bottleNeckIdentifyPSPの繰り返しの用意\n",
    "        for i in range(n_blocks - 1):\n",
    "            self.add_module(\n",
    "                \"block\" + str(i+2),\n",
    "                bottleNeckIdentifyPSP(\n",
    "                    out_channels, mid_channels, stride, dilation)\n",
    "            )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "class conv2DBatchNorm(nn.Module):\n",
    "    def __init__(self, in_channels, out_channels, kernel_size, stride, padding, dilation, bias):\n",
    "        super(conv2DBatchNorm, self).__init__()\n",
    "        self.conv = nn.Conv2d(in_channels, out_channels,\n",
    "                              kernel_size, stride, padding, dilation, bias=bias)\n",
    "        self.batchnorm = nn.BatchNorm2d(out_channels)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.conv(x)\n",
    "        outputs = self.batchnorm(x)\n",
    "\n",
    "        return outputs\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class bottleNeckPSP(nn.Module):\n",
    "    def __init__(self, in_channels, mid_channels, out_channels, stride, dilation):\n",
    "        super(bottleNeckPSP, self).__init__()\n",
    "\n",
    "        self.cbr_1 = conv2DBatchNormRelu(\n",
    "            in_channels, mid_channels, kernel_size=1, stride=1, padding=0, dilation=1, bias=False)\n",
    "        self.cbr_2 = conv2DBatchNormRelu(\n",
    "            mid_channels, mid_channels, kernel_size=3, stride=stride, padding=dilation, dilation=dilation, bias=False)\n",
    "        self.cb_3 = conv2DBatchNorm(\n",
    "            mid_channels, out_channels, kernel_size=1, stride=1, padding=0, dilation=1, bias=False)\n",
    "\n",
    "        # スキップ結合\n",
    "        self.cb_residual = conv2DBatchNorm(\n",
    "            in_channels, out_channels, kernel_size=1, stride=stride, padding=0, dilation=1, bias=False)\n",
    "\n",
    "        self.relu = nn.ReLU(inplace=True)\n",
    "\n",
    "    def forward(self, x):\n",
    "        conv = self.cb_3(self.cbr_2(self.cbr_1(x)))\n",
    "        residual = self.cb_residual(x)\n",
    "        return self.relu(conv + residual)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "class bottleNeckIdentifyPSP(nn.Module):\n",
    "    def __init__(self, in_channels, mid_channels, stride, dilation):\n",
    "        super(bottleNeckIdentifyPSP, self).__init__()\n",
    "\n",
    "        self.cbr_1 = conv2DBatchNormRelu(\n",
    "            in_channels, mid_channels, kernel_size=1, stride=1, padding=0, dilation=1, bias=False)\n",
    "        self.cbr_2 = conv2DBatchNormRelu(\n",
    "            mid_channels, mid_channels, kernel_size=3, stride=1, padding=dilation, dilation=dilation, bias=False)\n",
    "        self.cb_3 = conv2DBatchNorm(\n",
    "            mid_channels, in_channels, kernel_size=1, stride=1, padding=0, dilation=1, bias=False)\n",
    "        self.relu = nn.ReLU(inplace=True)\n",
    "\n",
    "    def forward(self, x):\n",
    "        conv = self.cb_3(self.cbr_2(self.cbr_1(x)))\n",
    "        residual = x\n",
    "        return self.relu(conv + residual)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.5 Pyramid Poolingモジュール"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PyramidPooling(nn.Module):\n",
    "    def __init__(self, in_channels, pool_sizes, height, width):\n",
    "        super(PyramidPooling, self).__init__()\n",
    "\n",
    "        # forwardで使用する画像サイズ\n",
    "        self.height = height\n",
    "        self.width = width\n",
    "\n",
    "        # 各畳み込み層の出力チャネル数\n",
    "        out_channels = int(in_channels / len(pool_sizes))\n",
    "\n",
    "        # 各畳み込み層を作成\n",
    "        # この実装方法は愚直すぎてfor文で書きたいところですが、分かりやすさを優先しています\n",
    "        # pool_sizes: [6, 3, 2, 1]\n",
    "        self.avpool_1 = nn.AdaptiveAvgPool2d(output_size=pool_sizes[0])\n",
    "        self.cbr_1 = conv2DBatchNormRelu(\n",
    "            in_channels, out_channels, kernel_size=1, stride=1, padding=0, dilation=1, bias=False)\n",
    "\n",
    "        self.avpool_2 = nn.AdaptiveAvgPool2d(output_size=pool_sizes[1])\n",
    "        self.cbr_2 = conv2DBatchNormRelu(\n",
    "            in_channels, out_channels, kernel_size=1, stride=1, padding=0, dilation=1, bias=False)\n",
    "\n",
    "        self.avpool_3 = nn.AdaptiveAvgPool2d(output_size=pool_sizes[2])\n",
    "        self.cbr_3 = conv2DBatchNormRelu(\n",
    "            in_channels, out_channels, kernel_size=1, stride=1, padding=0, dilation=1, bias=False)\n",
    "\n",
    "        self.avpool_4 = nn.AdaptiveAvgPool2d(output_size=pool_sizes[3])\n",
    "        self.cbr_4 = conv2DBatchNormRelu(\n",
    "            in_channels, out_channels, kernel_size=1, stride=1, padding=0, dilation=1, bias=False)\n",
    "\n",
    "    def forward(self, x):\n",
    "\n",
    "        out1 = self.cbr_1(self.avpool_1(x))\n",
    "        out1 = F.interpolate(out1, size=(\n",
    "            self.height, self.width), mode=\"bilinear\", align_corners=True)\n",
    "\n",
    "        out2 = self.cbr_2(self.avpool_2(x))\n",
    "        out2 = F.interpolate(out2, size=(\n",
    "            self.height, self.width), mode=\"bilinear\", align_corners=True)\n",
    "\n",
    "        out3 = self.cbr_3(self.avpool_3(x))\n",
    "        out3 = F.interpolate(out3, size=(\n",
    "            self.height, self.width), mode=\"bilinear\", align_corners=True)\n",
    "\n",
    "        out4 = self.cbr_4(self.avpool_4(x))\n",
    "        out4 = F.interpolate(out4, size=(\n",
    "            self.height, self.width), mode=\"bilinear\", align_corners=True)\n",
    "\n",
    "        # 最終的に結合させる、dim=1でチャネル数の次元で結合\n",
    "        output = torch.cat([x, out1, out2, out3, out4], dim=1)\n",
    "\n",
    "        return output\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.6 Decoder、AuxLossモジュール"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DecodePSPFeature(nn.Module):\n",
    "    def __init__(self, height, width, n_classes):\n",
    "        super(DecodePSPFeature, self).__init__()\n",
    "\n",
    "        # forwardで使用する画像サイズ\n",
    "        self.height = height\n",
    "        self.width = width\n",
    "\n",
    "        self.cbr = conv2DBatchNormRelu(\n",
    "            in_channels=4096, out_channels=512, kernel_size=3, stride=1, padding=1, dilation=1, bias=False)\n",
    "        self.dropout = nn.Dropout2d(p=0.1)\n",
    "        self.classification = nn.Conv2d(\n",
    "            in_channels=512, out_channels=n_classes, kernel_size=1, stride=1, padding=0)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.cbr(x)\n",
    "        x = self.dropout(x)\n",
    "        x = self.classification(x)\n",
    "        output = F.interpolate(\n",
    "            x, size=(self.height, self.width), mode=\"bilinear\", align_corners=True)\n",
    "\n",
    "        return output\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "class AuxiliaryPSPlayers(nn.Module):\n",
    "    def __init__(self, in_channels, height, width, n_classes):\n",
    "        super(AuxiliaryPSPlayers, self).__init__()\n",
    "\n",
    "        # forwardで使用する画像サイズ\n",
    "        self.height = height\n",
    "        self.width = width\n",
    "\n",
    "        self.cbr = conv2DBatchNormRelu(\n",
    "            in_channels=in_channels, out_channels=256, kernel_size=3, stride=1, padding=1, dilation=1, bias=False)\n",
    "        self.dropout = nn.Dropout2d(p=0.1)\n",
    "        self.classification = nn.Conv2d(\n",
    "            in_channels=256, out_channels=n_classes, kernel_size=1, stride=1, padding=0)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.cbr(x)\n",
    "        x = self.dropout(x)\n",
    "        x = self.classification(x)\n",
    "        output = F.interpolate(\n",
    "            x, size=(self.height, self.width), mode=\"bilinear\", align_corners=True)\n",
    "\n",
    "        return output\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 動作確認"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PSPNet(\n",
       "  (feature_conv): FeatureMap_convolution(\n",
       "    (cbnr_1): conv2DBatchNormRelu(\n",
       "      (conv): Conv2d(3, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "      (batchnorm): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (cbnr_2): conv2DBatchNormRelu(\n",
       "      (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (batchnorm): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (cbnr_3): conv2DBatchNormRelu(\n",
       "      (conv): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (batchnorm): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
       "  )\n",
       "  (feature_res_1): ResidualBlockPSP(\n",
       "    (block1): bottleNeckPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(128, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (cb_residual): conv2DBatchNorm(\n",
       "        (conv): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block2): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block3): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "  )\n",
       "  (feature_res_2): ResidualBlockPSP(\n",
       "    (block1): bottleNeckPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (cb_residual): conv2DBatchNorm(\n",
       "        (conv): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block2): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block3): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block4): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "  )\n",
       "  (feature_dilated_res_1): ResidualBlockPSP(\n",
       "    (block1): bottleNeckPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (cb_residual): conv2DBatchNorm(\n",
       "        (conv): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block2): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block3): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block4): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block5): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block6): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)\n",
       "        (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "  )\n",
       "  (feature_dilated_res_2): ResidualBlockPSP(\n",
       "    (block1): bottleNeckPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(4, 4), dilation=(4, 4), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (cb_residual): conv2DBatchNorm(\n",
       "        (conv): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block2): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(4, 4), dilation=(4, 4), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (block3): bottleNeckIdentifyPSP(\n",
       "      (cbr_1): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cbr_2): conv2DBatchNormRelu(\n",
       "        (conv): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(4, 4), dilation=(4, 4), bias=False)\n",
       "        (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace)\n",
       "      )\n",
       "      (cb_3): conv2DBatchNorm(\n",
       "        (conv): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (batchnorm): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "  )\n",
       "  (pyramid_pooling): PyramidPooling(\n",
       "    (avpool_1): AdaptiveAvgPool2d(output_size=6)\n",
       "    (cbr_1): conv2DBatchNormRelu(\n",
       "      (conv): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (avpool_2): AdaptiveAvgPool2d(output_size=3)\n",
       "    (cbr_2): conv2DBatchNormRelu(\n",
       "      (conv): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (avpool_3): AdaptiveAvgPool2d(output_size=2)\n",
       "    (cbr_3): conv2DBatchNormRelu(\n",
       "      (conv): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (avpool_4): AdaptiveAvgPool2d(output_size=1)\n",
       "    (cbr_4): conv2DBatchNormRelu(\n",
       "      (conv): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "  )\n",
       "  (decode_feature): DecodePSPFeature(\n",
       "    (cbr): conv2DBatchNormRelu(\n",
       "      (conv): Conv2d(4096, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (batchnorm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (dropout): Dropout2d(p=0.1)\n",
       "    (classification): Conv2d(512, 21, kernel_size=(1, 1), stride=(1, 1))\n",
       "  )\n",
       "  (aux): AuxiliaryPSPlayers(\n",
       "    (cbr): conv2DBatchNormRelu(\n",
       "      (conv): Conv2d(1024, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (batchnorm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace)\n",
       "    )\n",
       "    (dropout): Dropout2d(p=0.1)\n",
       "    (classification): Conv2d(256, 21, kernel_size=(1, 1), stride=(1, 1))\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# モデルの定義\n",
    "net = PSPNet(n_classes=21)\n",
    "net"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(tensor([[[[-2.1243e-01, -1.6445e-01, -1.1647e-01,  ..., -2.3913e-01,\n",
      "           -2.6464e-01, -2.9015e-01],\n",
      "          [-2.1309e-01, -1.6993e-01, -1.2676e-01,  ..., -2.2230e-01,\n",
      "           -2.4159e-01, -2.6088e-01],\n",
      "          [-2.1375e-01, -1.7540e-01, -1.3705e-01,  ..., -2.0547e-01,\n",
      "           -2.1854e-01, -2.3160e-01],\n",
      "          ...,\n",
      "          [-4.6660e-01, -4.2315e-01, -3.7970e-01,  ..., -1.9586e-01,\n",
      "           -1.6463e-01, -1.3339e-01],\n",
      "          [-4.3864e-01, -4.0260e-01, -3.6657e-01,  ..., -2.1617e-01,\n",
      "           -1.8173e-01, -1.4728e-01],\n",
      "          [-4.1068e-01, -3.8206e-01, -3.5344e-01,  ..., -2.3648e-01,\n",
      "           -1.9883e-01, -1.6117e-01]],\n",
      "\n",
      "         [[ 2.8644e-02,  2.6192e-02,  2.3740e-02,  ..., -2.0555e-01,\n",
      "           -2.1533e-01, -2.2510e-01],\n",
      "          [ 1.5821e-02,  1.7298e-02,  1.8775e-02,  ..., -1.8567e-01,\n",
      "           -1.9531e-01, -2.0495e-01],\n",
      "          [ 2.9983e-03,  8.4041e-03,  1.3810e-02,  ..., -1.6579e-01,\n",
      "           -1.7530e-01, -1.8481e-01],\n",
      "          ...,\n",
      "          [-2.7596e-01, -2.9349e-01, -3.1102e-01,  ..., -2.1052e-01,\n",
      "           -1.7620e-01, -1.4187e-01],\n",
      "          [-3.0834e-01, -3.2018e-01, -3.3202e-01,  ..., -2.1473e-01,\n",
      "           -1.7036e-01, -1.2599e-01],\n",
      "          [-3.4072e-01, -3.4687e-01, -3.5302e-01,  ..., -2.1893e-01,\n",
      "           -1.6452e-01, -1.1010e-01]],\n",
      "\n",
      "         [[-6.4997e-02, -6.1410e-02, -5.7823e-02,  ..., -9.3014e-03,\n",
      "            2.5667e-03,  1.4435e-02],\n",
      "          [-7.0839e-02, -6.6547e-02, -6.2255e-02,  ..., -1.3854e-02,\n",
      "            3.0930e-04,  1.4473e-02],\n",
      "          [-7.6681e-02, -7.1684e-02, -6.6687e-02,  ..., -1.8407e-02,\n",
      "           -1.9481e-03,  1.4511e-02],\n",
      "          ...,\n",
      "          [-1.5778e-01, -1.6529e-01, -1.7280e-01,  ..., -1.2837e-01,\n",
      "           -1.4361e-01, -1.5885e-01],\n",
      "          [-1.5816e-01, -1.6520e-01, -1.7224e-01,  ..., -1.3491e-01,\n",
      "           -1.5413e-01, -1.7336e-01],\n",
      "          [-1.5853e-01, -1.6510e-01, -1.7167e-01,  ..., -1.4145e-01,\n",
      "           -1.6465e-01, -1.8786e-01]],\n",
      "\n",
      "         ...,\n",
      "\n",
      "         [[ 2.5850e-01,  2.8503e-01,  3.1156e-01,  ...,  1.4771e-01,\n",
      "            1.5106e-01,  1.5441e-01],\n",
      "          [ 2.6579e-01,  2.8786e-01,  3.0993e-01,  ...,  1.0311e-01,\n",
      "            1.0916e-01,  1.1521e-01],\n",
      "          [ 2.7308e-01,  2.9069e-01,  3.0830e-01,  ...,  5.8518e-02,\n",
      "            6.7269e-02,  7.6020e-02],\n",
      "          ...,\n",
      "          [ 2.1242e-01,  1.9636e-01,  1.8030e-01,  ...,  6.9252e-02,\n",
      "            5.8550e-02,  4.7847e-02],\n",
      "          [ 1.8321e-01,  1.7115e-01,  1.5909e-01,  ...,  7.0830e-02,\n",
      "            6.1274e-02,  5.1717e-02],\n",
      "          [ 1.5400e-01,  1.4593e-01,  1.3787e-01,  ...,  7.2408e-02,\n",
      "            6.3998e-02,  5.5587e-02]],\n",
      "\n",
      "         [[-1.7526e-01, -1.4890e-01, -1.2254e-01,  ...,  1.6861e-01,\n",
      "            1.4851e-01,  1.2841e-01],\n",
      "          [-1.6340e-01, -1.4167e-01, -1.1994e-01,  ...,  1.4103e-01,\n",
      "            1.2482e-01,  1.0861e-01],\n",
      "          [-1.5153e-01, -1.3444e-01, -1.1734e-01,  ...,  1.1344e-01,\n",
      "            1.0112e-01,  8.8809e-02],\n",
      "          ...,\n",
      "          [ 1.5739e-01,  1.2155e-01,  8.5719e-02,  ...,  6.4970e-02,\n",
      "            6.8426e-02,  7.1882e-02],\n",
      "          [ 1.1474e-01,  7.9551e-02,  4.4357e-02,  ...,  4.2494e-02,\n",
      "            4.7192e-02,  5.1890e-02],\n",
      "          [ 7.2098e-02,  3.7547e-02,  2.9950e-03,  ...,  2.0018e-02,\n",
      "            2.5958e-02,  3.1897e-02]],\n",
      "\n",
      "         [[ 2.9752e-02,  2.5839e-02,  2.1927e-02,  ..., -1.3459e-02,\n",
      "            7.2033e-03,  2.7866e-02],\n",
      "          [ 1.4661e-02,  1.2977e-02,  1.1293e-02,  ...,  3.9192e-03,\n",
      "            1.9902e-02,  3.5885e-02],\n",
      "          [-4.2949e-04,  1.1453e-04,  6.5855e-04,  ...,  2.1298e-02,\n",
      "            3.2601e-02,  4.3903e-02],\n",
      "          ...,\n",
      "          [ 8.6814e-03,  5.0448e-03,  1.4083e-03,  ..., -3.2342e-02,\n",
      "           -5.2540e-03,  2.1834e-02],\n",
      "          [ 4.1567e-02,  3.6202e-02,  3.0838e-02,  ..., -3.5315e-02,\n",
      "           -7.8293e-03,  1.9657e-02],\n",
      "          [ 7.4452e-02,  6.7360e-02,  6.0269e-02,  ..., -3.8289e-02,\n",
      "           -1.0405e-02,  1.7479e-02]]],\n",
      "\n",
      "\n",
      "        [[[-1.1010e-01, -1.4108e-01, -1.7207e-01,  ..., -2.0727e-01,\n",
      "           -2.0171e-01, -1.9616e-01],\n",
      "          [-1.3206e-01, -1.5373e-01, -1.7539e-01,  ..., -2.2255e-01,\n",
      "           -2.1751e-01, -2.1248e-01],\n",
      "          [-1.5402e-01, -1.6637e-01, -1.7871e-01,  ..., -2.3783e-01,\n",
      "           -2.3331e-01, -2.2880e-01],\n",
      "          ...,\n",
      "          [-3.1476e-01, -3.2234e-01, -3.2992e-01,  ..., -2.6046e-01,\n",
      "           -2.5433e-01, -2.4820e-01],\n",
      "          [-3.4108e-01, -3.4881e-01, -3.5653e-01,  ..., -2.6120e-01,\n",
      "           -2.4779e-01, -2.3439e-01],\n",
      "          [-3.6739e-01, -3.7527e-01, -3.8315e-01,  ..., -2.6194e-01,\n",
      "           -2.4126e-01, -2.2058e-01]],\n",
      "\n",
      "         [[-1.9026e-01, -1.3061e-01, -7.0968e-02,  ..., -4.7564e-02,\n",
      "           -3.8661e-02, -2.9757e-02],\n",
      "          [-1.8707e-01, -1.3500e-01, -8.2941e-02,  ..., -5.4379e-02,\n",
      "           -4.0488e-02, -2.6596e-02],\n",
      "          [-1.8388e-01, -1.3940e-01, -9.4913e-02,  ..., -6.1194e-02,\n",
      "           -4.2314e-02, -2.3435e-02],\n",
      "          ...,\n",
      "          [-1.7651e-01, -1.7887e-01, -1.8123e-01,  ..., -1.9203e-01,\n",
      "           -1.7747e-01, -1.6291e-01],\n",
      "          [-1.8278e-01, -1.8548e-01, -1.8818e-01,  ..., -1.8861e-01,\n",
      "           -1.6983e-01, -1.5104e-01],\n",
      "          [-1.8906e-01, -1.9209e-01, -1.9512e-01,  ..., -1.8520e-01,\n",
      "           -1.6218e-01, -1.3917e-01]],\n",
      "\n",
      "         [[-2.7928e-01, -2.7132e-01, -2.6336e-01,  ..., -4.5442e-02,\n",
      "           -2.9658e-02, -1.3874e-02],\n",
      "          [-2.6422e-01, -2.5770e-01, -2.5118e-01,  ..., -6.5181e-02,\n",
      "           -4.8775e-02, -3.2369e-02],\n",
      "          [-2.4916e-01, -2.4408e-01, -2.3901e-01,  ..., -8.4920e-02,\n",
      "           -6.7892e-02, -5.0864e-02],\n",
      "          ...,\n",
      "          [-8.6330e-02, -1.2361e-01, -1.6089e-01,  ..., -1.4620e-01,\n",
      "           -1.7705e-01, -2.0790e-01],\n",
      "          [-5.2899e-02, -9.1973e-02, -1.3105e-01,  ..., -1.4284e-01,\n",
      "           -1.7094e-01, -1.9904e-01],\n",
      "          [-1.9467e-02, -6.0335e-02, -1.0120e-01,  ..., -1.3949e-01,\n",
      "           -1.6484e-01, -1.9019e-01]],\n",
      "\n",
      "         ...,\n",
      "\n",
      "         [[ 2.8684e-01,  3.2318e-01,  3.5952e-01,  ...,  3.0021e-01,\n",
      "            3.4302e-01,  3.8583e-01],\n",
      "          [ 2.7541e-01,  3.0683e-01,  3.3825e-01,  ...,  2.6761e-01,\n",
      "            3.0774e-01,  3.4788e-01],\n",
      "          [ 2.6398e-01,  2.9048e-01,  3.1698e-01,  ...,  2.3501e-01,\n",
      "            2.7247e-01,  3.0992e-01],\n",
      "          ...,\n",
      "          [ 2.7566e-01,  2.5930e-01,  2.4295e-01,  ...,  1.9233e-01,\n",
      "            1.7965e-01,  1.6697e-01],\n",
      "          [ 3.0450e-01,  2.8263e-01,  2.6076e-01,  ...,  1.8257e-01,\n",
      "            1.6574e-01,  1.4891e-01],\n",
      "          [ 3.3335e-01,  3.0597e-01,  2.7858e-01,  ...,  1.7282e-01,\n",
      "            1.5184e-01,  1.3085e-01]],\n",
      "\n",
      "         [[-1.9905e-01, -2.0445e-01, -2.0985e-01,  ..., -3.5642e-02,\n",
      "           -3.3235e-02, -3.0829e-02],\n",
      "          [-1.5288e-01, -1.6493e-01, -1.7698e-01,  ..., -3.1504e-02,\n",
      "           -2.9621e-02, -2.7737e-02],\n",
      "          [-1.0670e-01, -1.2541e-01, -1.4411e-01,  ..., -2.7367e-02,\n",
      "           -2.6006e-02, -2.4645e-02],\n",
      "          ...,\n",
      "          [ 3.5657e-02,  3.2331e-02,  2.9005e-02,  ...,  1.7088e-01,\n",
      "            1.8569e-01,  2.0050e-01],\n",
      "          [ 6.1775e-02,  5.6678e-02,  5.1580e-02,  ...,  1.5884e-01,\n",
      "            1.7235e-01,  1.8585e-01],\n",
      "          [ 8.7894e-02,  8.1025e-02,  7.4155e-02,  ...,  1.4681e-01,\n",
      "            1.5900e-01,  1.7120e-01]],\n",
      "\n",
      "         [[ 1.6555e-01,  1.4397e-01,  1.2238e-01,  ...,  5.7056e-02,\n",
      "            5.0822e-02,  4.4589e-02],\n",
      "          [ 1.5046e-01,  1.3132e-01,  1.1218e-01,  ...,  5.1962e-02,\n",
      "            5.0005e-02,  4.8047e-02],\n",
      "          [ 1.3537e-01,  1.1867e-01,  1.0198e-01,  ...,  4.6869e-02,\n",
      "            4.9187e-02,  5.1505e-02],\n",
      "          ...,\n",
      "          [ 1.5112e-02,  1.1025e-02,  6.9380e-03,  ...,  9.6608e-02,\n",
      "            1.0490e-01,  1.1319e-01],\n",
      "          [ 5.8697e-02,  4.8298e-02,  3.7899e-02,  ...,  1.2636e-01,\n",
      "            1.3415e-01,  1.4193e-01],\n",
      "          [ 1.0228e-01,  8.5571e-02,  6.8860e-02,  ...,  1.5611e-01,\n",
      "            1.6340e-01,  1.7068e-01]]]], grad_fn=<UpsampleBilinear2DBackward>), tensor([[[[ 0.2819,  0.2819,  0.2818,  ...,  0.4269,  0.4141,  0.4013],\n",
      "          [ 0.2823,  0.2811,  0.2798,  ...,  0.4238,  0.4147,  0.4056],\n",
      "          [ 0.2826,  0.2802,  0.2778,  ...,  0.4207,  0.4153,  0.4098],\n",
      "          ...,\n",
      "          [ 0.1675,  0.1690,  0.1705,  ...,  0.4334,  0.4354,  0.4374],\n",
      "          [ 0.1764,  0.1771,  0.1778,  ...,  0.4294,  0.4284,  0.4274],\n",
      "          [ 0.1853,  0.1852,  0.1851,  ...,  0.4255,  0.4215,  0.4175]],\n",
      "\n",
      "         [[ 0.0669,  0.0952,  0.1236,  ...,  0.2217,  0.2291,  0.2366],\n",
      "          [ 0.0961,  0.1196,  0.1430,  ...,  0.2198,  0.2231,  0.2265],\n",
      "          [ 0.1254,  0.1439,  0.1623,  ...,  0.2178,  0.2171,  0.2164],\n",
      "          ...,\n",
      "          [ 0.2566,  0.2194,  0.1821,  ..., -0.0625, -0.0413, -0.0201],\n",
      "          [ 0.2579,  0.2170,  0.1762,  ..., -0.0522, -0.0341, -0.0160],\n",
      "          [ 0.2591,  0.2147,  0.1703,  ..., -0.0419, -0.0269, -0.0119]],\n",
      "\n",
      "         [[ 0.0232,  0.0239,  0.0245,  ...,  0.2071,  0.2367,  0.2664],\n",
      "          [ 0.0436,  0.0416,  0.0395,  ...,  0.2328,  0.2650,  0.2972],\n",
      "          [ 0.0640,  0.0593,  0.0545,  ...,  0.2585,  0.2933,  0.3281],\n",
      "          ...,\n",
      "          [ 0.2520,  0.2803,  0.3086,  ...,  0.2901,  0.2714,  0.2526],\n",
      "          [ 0.2357,  0.2645,  0.2934,  ...,  0.2737,  0.2478,  0.2218],\n",
      "          [ 0.2194,  0.2488,  0.2781,  ...,  0.2572,  0.2242,  0.1911]],\n",
      "\n",
      "         ...,\n",
      "\n",
      "         [[-0.0667, -0.0722, -0.0777,  ..., -0.2804, -0.2898, -0.2992],\n",
      "          [-0.0932, -0.0967, -0.1002,  ..., -0.2846, -0.2910, -0.2974],\n",
      "          [-0.1197, -0.1212, -0.1226,  ..., -0.2888, -0.2921, -0.2955],\n",
      "          ...,\n",
      "          [-0.2331, -0.2256, -0.2181,  ..., -0.1179, -0.1236, -0.1293],\n",
      "          [-0.2333, -0.2253, -0.2173,  ..., -0.1076, -0.1234, -0.1391],\n",
      "          [-0.2334, -0.2249, -0.2165,  ..., -0.0974, -0.1231, -0.1489]],\n",
      "\n",
      "         [[-0.1807, -0.1587, -0.1366,  ..., -0.0523, -0.0704, -0.0885],\n",
      "          [-0.1831, -0.1605, -0.1379,  ..., -0.0659, -0.0812, -0.0964],\n",
      "          [-0.1856, -0.1624, -0.1393,  ..., -0.0796, -0.0919, -0.1043],\n",
      "          ...,\n",
      "          [-0.1956, -0.1999, -0.2042,  ..., -0.0088, -0.0057, -0.0027],\n",
      "          [-0.2373, -0.2431, -0.2489,  ..., -0.0220, -0.0208, -0.0196],\n",
      "          [-0.2791, -0.2863, -0.2936,  ..., -0.0351, -0.0358, -0.0365]],\n",
      "\n",
      "         [[-0.2828, -0.2804, -0.2780,  ..., -0.1125, -0.1102, -0.1080],\n",
      "          [-0.2861, -0.2840, -0.2819,  ..., -0.1347, -0.1316, -0.1285],\n",
      "          [-0.2894, -0.2875, -0.2857,  ..., -0.1569, -0.1530, -0.1491],\n",
      "          ...,\n",
      "          [-0.0790, -0.0862, -0.0935,  ..., -0.1513, -0.1634, -0.1755],\n",
      "          [-0.0915, -0.0952, -0.0989,  ..., -0.1360, -0.1424, -0.1488],\n",
      "          [-0.1040, -0.1042, -0.1044,  ..., -0.1206, -0.1214, -0.1221]]],\n",
      "\n",
      "\n",
      "        [[[ 0.2852,  0.2789,  0.2726,  ...,  0.3991,  0.4126,  0.4260],\n",
      "          [ 0.3069,  0.2971,  0.2873,  ...,  0.4208,  0.4370,  0.4532],\n",
      "          [ 0.3287,  0.3154,  0.3020,  ...,  0.4426,  0.4615,  0.4804],\n",
      "          ...,\n",
      "          [ 0.2972,  0.2758,  0.2545,  ...,  0.3563,  0.3848,  0.4134],\n",
      "          [ 0.3402,  0.3115,  0.2829,  ...,  0.3738,  0.4041,  0.4344],\n",
      "          [ 0.3832,  0.3472,  0.3112,  ...,  0.3914,  0.4234,  0.4555]],\n",
      "\n",
      "         [[ 0.2126,  0.2243,  0.2360,  ...,  0.1720,  0.1868,  0.2016],\n",
      "          [ 0.1920,  0.2065,  0.2209,  ...,  0.1549,  0.1637,  0.1725],\n",
      "          [ 0.1715,  0.1886,  0.2058,  ...,  0.1379,  0.1407,  0.1435],\n",
      "          ...,\n",
      "          [ 0.1214,  0.1214,  0.1213,  ...,  0.1084,  0.1383,  0.1683],\n",
      "          [ 0.1346,  0.1292,  0.1238,  ...,  0.1264,  0.1647,  0.2030],\n",
      "          [ 0.1478,  0.1370,  0.1262,  ...,  0.1443,  0.1910,  0.2377]],\n",
      "\n",
      "         [[ 0.1862,  0.1947,  0.2032,  ...,  0.1885,  0.1683,  0.1480],\n",
      "          [ 0.2031,  0.2068,  0.2105,  ...,  0.2066,  0.1925,  0.1784],\n",
      "          [ 0.2200,  0.2188,  0.2177,  ...,  0.2247,  0.2167,  0.2088],\n",
      "          ...,\n",
      "          [ 0.2983,  0.3113,  0.3244,  ...,  0.2200,  0.1998,  0.1796],\n",
      "          [ 0.3555,  0.3661,  0.3767,  ...,  0.2036,  0.1803,  0.1570],\n",
      "          [ 0.4127,  0.4209,  0.4290,  ...,  0.1872,  0.1609,  0.1345]],\n",
      "\n",
      "         ...,\n",
      "\n",
      "         [[-0.1325, -0.1318, -0.1311,  ..., -0.1787, -0.1859, -0.1930],\n",
      "          [-0.1288, -0.1271, -0.1253,  ..., -0.1573, -0.1611, -0.1648],\n",
      "          [-0.1252, -0.1224, -0.1196,  ..., -0.1360, -0.1363, -0.1366],\n",
      "          ...,\n",
      "          [-0.1002, -0.1090, -0.1177,  ..., -0.0768, -0.0731, -0.0694],\n",
      "          [-0.1018, -0.1173, -0.1327,  ..., -0.0914, -0.0876, -0.0839],\n",
      "          [-0.1035, -0.1256, -0.1477,  ..., -0.1059, -0.1021, -0.0983]],\n",
      "\n",
      "         [[-0.0970, -0.1073, -0.1175,  ..., -0.0455, -0.0364, -0.0273],\n",
      "          [-0.0822, -0.0955, -0.1088,  ..., -0.0338, -0.0248, -0.0158],\n",
      "          [-0.0674, -0.0837, -0.1000,  ..., -0.0220, -0.0132, -0.0044],\n",
      "          ...,\n",
      "          [-0.0085,  0.0112,  0.0308,  ...,  0.0071,  0.0050,  0.0028],\n",
      "          [-0.0115,  0.0132,  0.0378,  ..., -0.0120, -0.0162, -0.0204],\n",
      "          [-0.0144,  0.0152,  0.0447,  ..., -0.0312, -0.0374, -0.0436]],\n",
      "\n",
      "         [[-0.4353, -0.4414, -0.4475,  ..., -0.2110, -0.2062, -0.2013],\n",
      "          [-0.3879, -0.3932, -0.3985,  ..., -0.2110, -0.2051, -0.1993],\n",
      "          [-0.3405, -0.3449, -0.3494,  ..., -0.2109, -0.2041, -0.1973],\n",
      "          ...,\n",
      "          [-0.2143, -0.2133, -0.2124,  ..., -0.2916, -0.3077, -0.3237],\n",
      "          [-0.2162, -0.2137, -0.2112,  ..., -0.2601, -0.2798, -0.2995],\n",
      "          [-0.2181, -0.2140, -0.2099,  ..., -0.2286, -0.2519, -0.2753]]]],\n",
      "       grad_fn=<UpsampleBilinear2DBackward>))\n"
     ]
    }
   ],
   "source": [
    "# ダミーデータの作成\n",
    "batch_size = 2\n",
    "dummy_img = torch.rand(batch_size, 3, 475, 475)\n",
    "\n",
    "# 計算\n",
    "outputs = net(dummy_img)\n",
    "print(outputs)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
